System and method for wireless ordering using speech recognition

ABSTRACT

Disclosed herein are systems, computer-implemented methods, and tangible computer-readable media for placing an order for a user. The method includes receiving a search from a user, identifying a product category based on the search, presenting to the user a general ordering screen based on the identified product category, selecting and activating a speech recognition grammar tuned for the identified product category, recognizing a first received user utterance with the activated tuned grammar to identify a vendor who offers items in the identified product category, recognizing a second received user utterance with the activated tuned grammar to identify a specific item from the identified vendor, and placing an order for the specific item with the identified vendor for the user. In one aspect, the method further offers to sell the user additional items ancillary to the specific item.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to product ordering and more specificallyto placing product orders for users based on speech recognition.

2. Introduction

With the widespread popularity of the Internet and secure ways toconduct commerce online, consumers purchase goods and services onlinewith increasing frequency. Online commerce differs from traditionalbrick-and-mortar commerce because consumers can browse to any webaddress with minimal difficulty. The geographic and spatial distancebetween businesses does not matter because each business's web presenceis literally at the consumer's fingertips. However, consumers are easilyoverwhelmed by so many options and may not know all the relevant optionsor even which options are relevant.

Often a consumer does not have a specific merchant or a specific item inmind, but just a category, such as pizza parlor, florist, or toy store.Searching for a category of merchants and making a purchase with one isa cumbersome and mismatched process involving opening a search engine,formulating a query, browsing through the results one by one until aparticular result is good enough. Each merchant has a differentinterface, separate user profiles, different product lines, etc. If theuser is not already familiar with a particular merchant, the user isless likely to visit that merchant in an online environment. In a mall,for example, a merchant's store front, decor, music, employees, etc. arean invitation for consumers to enter, browse, and purchase. In otherwords, merchants can control the environment to some extent to maketheir store more appealing. Consumers wander through the mall, often forleisure, and browse stores that interest them. Many of the real lifeaspects of shopping are absent in online commerce. For example, a lineof text in a list of search results does attract customers like a storewindow showing the latest fashions and advertising a sale or a pizzashop with a large picture of a slice of pizza and free samples.

Accordingly, what is needed in the art is an improved way for users topurchase goods and services from vendors in an easy-to-use, naturalinterface.

SUMMARY

Additional features and advantages of the invention will be set forth inthe description which follows, and in part will be obvious from thedescription, or may be learned by practice of the invention. Thefeatures and advantages of the invention may be realized and obtained bymeans of the instruments and combinations particularly pointed out inthe appended claims. These and other features of the present inventionwill become more fully apparent from the following description andappended claims, or may be learned by the practice of the invention asset forth herein.

Disclosed are systems, computer-implemented methods, and tangiblecomputer-readable media for placing an order for a user. The methodincludes receiving a search from a user, identifying a product categorybased on the search, presenting to the user a general ordering screenbased on the identified product category, selecting and activating aspeech recognition grammar tuned for the identified product category,recognizing a first received user utterance with the activated tunedgrammar to identify a vendor who offers items in the identified productcategory, recognizing a second received user utterance with theactivated tuned grammar to identify a specific item from the identifiedvendor, and placing an order for the specific item with the identifiedvendor for the user. In one aspect, the method further offers to sellthe user additional items ancillary to the specific item. In anotheraspect, the method further receives user billing information and paysfor the placed order using the received user billing information. Themethod can communicate to the user information regarding order pick up.The method can gather delivery instructions from the user andcommunicate the gathered delivery instructions to the identified vendor.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and otheradvantages and features of the invention can be obtained, a moreparticular description of the invention briefly described above will berendered by reference to specific embodiments thereof which areillustrated in the appended drawings. Understanding that these drawingsdepict only exemplary embodiments of the invention and are not thereforeto be considered to be limiting of its scope, the invention will bedescribed and explained with additional specificity and detail throughthe use of the accompanying drawings in which:

FIG. 1 illustrates an example system embodiment;

FIG. 2 illustrates an example method embodiment; and

FIG. 3 illustrates an example system for placing orders on behalf ofusers.

DETAILED DESCRIPTION

Various embodiments of the invention are discussed in detail below.While specific implementations are discussed, it should be understoodthat this is done for illustration purposes only. A person skilled inthe relevant art will recognize that other components and configurationsmay be used without parting from the spirit and scope of the invention.

With reference to FIG. 1, an exemplary system includes a general-purposecomputing device 100, including a processing unit (CPU) 120 and a systembus 110 that couples various system components including the systemmemory such as read only memory (ROM) 140 and random access memory (RAM)150 to the processing unit 120. Other system memory 130 may be availablefor use as well. It can be appreciated that the invention may operate ona computing device with more than one CPU 120 or on a group or clusterof computing devices networked together to provide greater processingcapability. A processing unit 120 can include a general purpose CPUcontrolled by software as well as a special-purpose processor. An IntelXeon LV L7345 processor is an example of a general purpose CPU which iscontrolled by software. Particular functionality may also be built intothe design of a separate computer chip. An STMicroelectronics STA013processor is an example of a special-purpose processor which decodes MP3audio files. Of course, a processing unit includes any general purposeCPU and a module configured to control the CPU as well as aspecial-purpose processor where software is effectively incorporatedinto the actual processor design. A processing unit may essentially be acompletely self-contained computing system, containing multiple cores orCPUs, a bus, memory controller, cache, etc. A multi-core processing unitmay be symmetric or asymmetric.

The system bus 110 may be any of several types of bus structuresincluding a memory bus or memory controller, a peripheral bus, and alocal bus using any of a variety of bus architectures. A basicinput/output (BIOS) stored in ROM 140 or the like, may provide the basicroutine that helps to transfer information between elements within thecomputing device 100, such as during start-up. The computing device 100further includes storage devices such as a hard disk drive 160, amagnetic disk drive, an optical disk drive, tape drive or the like. Thestorage device 160 is connected to the system bus 110 by a driveinterface. The drives and the associated computer readable media providenonvolatile storage of computer readable instructions, data structures,program modules and other data for the computing device 100. In oneaspect, a hardware module that performs a particular function includesthe software component stored in a tangible computer-readable medium inconnection with the necessary hardware components, such as the CPU, bus,display, and so forth, to carry out the function. The basic componentsare known to those of skill in the art and appropriate variations arecontemplated depending on the type of device, such as whether the deviceis a small, handheld computing device, a desktop computer, or a computerserver.

Although the exemplary environment described herein employs the harddisk, it should be appreciated by those skilled in the art that othertypes of computer readable media which can store data that areaccessible by a computer, such as magnetic cassettes, flash memorycards, digital versatile disks, cartridges, random access memories(RAMs), read only memory (ROM), a cable or wireless signal containing abit stream and the like, may also be used in the exemplary operatingenvironment.

To enable user interaction with the computing device 100, an inputdevice 190 represents any number of input mechanisms, such as amicrophone for speech, a touch-sensitive screen for gesture or graphicalinput, keyboard, mouse, motion input, speech and so forth. The input maybe used by the presenter to indicate the beginning of a speech searchquery. The device output 170 can also be one or more of a number ofoutput mechanisms known to those of skill in the art. In some instances,multimodal systems enable a user to provide multiple types of input tocommunicate with the computing device 100. The communications interface180 generally governs and manages the user input and system output.There is no restriction on the invention operating on any particularhardware arrangement and therefore the basic features here may easily besubstituted for improved hardware or firmware arrangements as they aredeveloped.

For clarity of explanation, the illustrative system embodiment ispresented as comprising individual functional blocks (includingfunctional blocks labeled as a “processor”). The functions these blocksrepresent may be provided through the use of either shared or dedicatedhardware, including, but not limited to, hardware capable of executingsoftware and hardware, such as a processor, that is purpose-built tooperate as an equivalent to software executing on a general purposeprocessor. For example the functions of one or more processors presentedin FIG. 1 may be provided by a single shared processor or multipleprocessors. (Use of the term “processor” should not be construed torefer exclusively to hardware capable of executing software.)Illustrative embodiments may comprise microprocessor and/or digitalsignal processor (DSP) hardware, read-only memory (ROM) for storingsoftware performing the operations discussed below, and random accessmemory (RAM) for storing results. Very large scale integration (VLSI)hardware embodiments, as well as custom VLSI circuitry in combinationwith a general purpose DSP circuit, may also be provided.

The logical operations of the various embodiments are implemented as:(1) a sequence of computer implemented steps, operations, or proceduresrunning on a programmable circuit within a general use computer, (2) asequence of computer implemented steps, operations, or proceduresrunning on a specific-use programmable circuit; and/or (3)interconnected machine modules or program engines within theprogrammable circuits.

Having disclosed some fundamental system components, the disclosureturns to the exemplary method embodiment for placing orders on behalf ofusers as illustrated in FIG. 2. For clarity, the method is discussed interms of a system configured to practice the method. The system firstreceives a search from a user (202). Some examples of a user search area voice command, a text entry in a field, and a selection of apredefined category. For example, a user of a wireless device can searchby saying the words “car rentals in Las Vegas”, while a user at acomputer can search by entering the text string “car rentals in LasVegas”. When the user enters speech at this initial search or at anylater time in the process, the system can verify the user's identitywith speaker recognition. For example, if the device is a cellularphone, the user's identity is somewhat known based on the cell phonenumber and/or other unique device identifiers. However, if someonesteals or “borrows” the cell phone and attempts to place an order, thesystem can frustrate their efforts by authenticating the imposter basedon speaker identification. This feature can be a setting the user candisable or enable to allow, for example, a spouse, child, close friend,or other trusted person to place orders using a cell phone that is nottheir own.

The system identifies a product category based on the search (204). Thesystem can identify product or service categories very broadly, verynarrowly, or somewhere in between. An example of a very broad productcategory is “pizza”. A narrower category is “pizza in Omaha, Nebr.”. Avery narrow category is “pizza within 5 minutes of downtown Omaha, Nebr.for under $15”. The system can define categories based on factors likelocation, price, quality, and reviews. In some cases, the search willinclude a location. In other cases where the product category is mostlikely restricted by location, such as almost any food category, thesystem can infer the location restriction when identifying the productcategory. One way to infer a location restriction is by GPS informationon a cellular phone.

The system presents to the user a general ordering screen based on theidentified product category (206). The general ordering screen can be avery simple, basic listing of products in the category. This screen canidentify vendors in the ordering screen or not. In one example of apizza category, the general ordering screen displays a picture of threedifferent sizes of pizza on a wireless device. When the user selects thelarge pizza, the other two disappear and the system displays a graphicfor each available topping. The user can say the name of each topping orthe user can drag them with a mouse, stylus, or finger in order to placethem on the picture of the large pizza. The general ordering screen canshow and dynamically update the estimated price or the lowest price forthe item based on vendors and products which match the identifiedproduct category.

The system selects and activates a speech recognition grammar tuned forthe identified product category (208). In a pizza ordering category, thesystem can use a speech recognition grammar tuned to recognize commonlyused words describing the product such as pepperoni, mushroom, olive,Canadian bacon, stuffed crust, large, medium, and small. The grammar canalso include common vendors such as Domino's and Pizza Hut. The systemcan select grammars from a predetermined list. In one aspect, the systemdynamically develops and refines grammars for categories based on whatusers actually say.

The system recognizes a first received user utterance with the activatedtuned grammar to identify a vendor who offers items in the identifiedproduct category (210). The system recognizes a second received userutterance with the activated tuned grammar to identify a specific itemfrom the identified vendor (212). The order in which the systemrecognizes these utterances is not important. The system can recognizeboth in one utterance. In the pizza example, the user can say “Largepepperoni pizza from Domino's” rather than saying two distinct, separateutterances.

In one variation, the system offers to sell the user additional itemsancillary to the specific item. In the example of ordering pizza, thesystem can offer to sell the user breadsticks, a desert pizza, softdrinks, or garlic bread. The system can display images of theseancillary items to the user as they are offered to provide an experiencesimilar to going to the pizza shop and seeing the fresh breadstickspulled out of the oven.

The system places an order for the specific item with the identifiedvendor for the user (214). The system can place the order using avariety of methods. For example, the system can send a text message oremail to the identified vendor. The system can use speech synthesis tocall the vendor and place the order. The order can include an indicationthat it was placed on behalf of the user. In one aspect, the systemrequests a receipt from the vendor. When the system receives thereceipt, the system notifies the user that the order has been received.

Some vendors require payment in advance before pickup or delivery. Inthese cases, the system can receive user billing information and pay forthe placed order using the received user billing information. The usercan enter credit card information or Paypal account information in auser profile for ordering pizza, which the system uses to pay pizzavendors. In one aspect, the system retains a portion of the payment inexchange for directing the user to purchase from the vendor. In anotheraspect, the system further communicates to the user informationregarding order pick up or gathers delivery instructions from the userand communicates the gathered delivery instructions to the identifiedvendor.

The system can retrieve a user profile associated with the verifieduser. One way the system can use the retrieved user profile is to ask ifthe user wants to order a commonly ordered item from a commonly usedvendor in the order history and place an order for the commonly ordereditem from the commonly used vendor if the user responds positively. Forexample, if the user has ordered a large mushroom pizza from Papa John'sin eight of the last ten orders, the system can provide a prompt at thegeneral ordering screen (or elsewhere) offering “the usual”, which is alarge mushroom pizza from Papa John's.

FIG. 3 illustrates an example system for placing orders on behalf ofusers. The system 300 can receive communications from users 302 at aserver 304. The communications can be multimodal. The communications caninclude, for example, voice commands, gestures, mouse clicks, and buttonpresses. The communications can be searches. For instance, a user cansay “search”, then type in a category of things to search for, such as“fast food”. The server 304 identifies a product category based on theuser communications. The server 304 retrieves and activates acategory-specific grammar 310 and retrieves a group of vendors 308offering goods or services in the category. The server can restrict oreliminate certain vendors from consideration based on the categoryand/or a user profile 306. For example, if the category is fast food,the server 304 can eliminate vendors who are more than 5 miles away. Ifthe user indicates she will be in a particular part of town later thatevening, the profile can restrict vendors to those in that particularpart of town. The user profile can further include information regardingprevious orders, commonly ordered items, common vendors, amountspreviously spent, and specific item preferences (such as the userprefers pizzas with sausage but no olives, or the user prefers dressesfrom department stores rather than discount outlets).

The server presents a general ordering screen to the user. The servercan present the ordering screen to the user on a mobile device, such asa smartphone, or on a device unrelated to the call, such as a televisionor digital photo picture frame. The system recognizes user speech withthe category-specific grammar. The system can optimize category-specificgrammars to recognize particular commonly used words and/or phrases inthe category-specific domain. For instance, a grammar for orderingtake-out Chinese food can be optimized to recognize words and phraseslike “fried rice”, “chow mein”, and “General Tso's chicken”. A grammarfor ordering a rental car can be optimized to recognize words andphrases like “all wheel drive”, “Chevrolet Cobalt”, and the days of theweek. The user identifies to the server 304 a product and a vendor usingspeech input. The server 304 recognizes the speech using thecategory-specific grammar 310 and places an order with the identifiedvendor. The server 304 can also offer or solicit to the user additionalor ancillary items on the screen or verbally. In the case of pizza, theserver 304 can offer breadsticks, mozzarella sticks, a soft drink, or a2 for 1 pizza deal. In the case of a rental car, the server 304 canoffer a discounted upgrade to a larger or more luxurious vehicle.

The server 304 can pay the vendor automatically using paymentinformation in the user profile 306, such as a credit card. The server304 can also gather payment information from the user for the currenttransaction and forward that information to the appropriate vendor 308.The server 304 can also receive payment from the user then pay thevendor without referring to any individual payment information of theuser. In some cases, the server 304 retains a portion of the fee inexchange for directing the user to the vendor. The server 304 can alsogather delivery instructions from the user or instruct the user where topick up the product.

One vendor can host the server 304 and ensure that their product orservice is always listed in comparison with other vendors, even if othervendors occasionally offer better prices or value. This approach exposesthe vendor's brand to more potential customers and establishes an air ofopenness which can win over customers. Progressive Insurance currentlyoperates using a similar model in the insurance industry by showingcomparable rate quotes from competitor insurance companies. A vendor,such as a pizza place, can benefit from showing other vendors' offeringsbecause they can track which competitors customers eventually patronizeand tailor their own offerings to be more competitive or moredistinctive.

Embodiments within the scope of the present invention may also includecomputer-readable media for carrying or having computer-executableinstructions or data structures stored thereon. Such computer-readablemedia can be any available media that can be accessed by a generalpurpose or special purpose computer, including the functional design ofany special purpose processor as discussed above. By way of example, andnot limitation, such computer-readable media can comprise RAM, ROM,EEPROM, CD-ROM or other optical disk storage, magnetic disk storage orother magnetic storage devices, or any other medium which can be used tocarry or store desired program code means in the form ofcomputer-executable instructions, data structures, or processor chipdesign. When information is transferred or provided over a network oranother communications connection (either hardwired, wireless, orcombination thereof) to a computer, the computer properly views theconnection as a computer-readable medium. Thus, any such connection isproperly termed a computer-readable medium. Combinations of the aboveshould also be included within the scope of the computer-readable media.

Computer-executable instructions include, for example, instructions anddata which cause a general purpose computer, special purpose computer,or special purpose processing device to perform a certain function orgroup of functions. Computer-executable instructions also includeprogram modules that are executed by computers in stand-alone or networkenvironments. Generally, program modules include routines, programs,objects, components, data structures, and the functions inherent in thedesign of special-purpose processors, etc. that perform particular tasksor implement particular abstract data types. Computer-executableinstructions, associated data structures, and program modules representexamples of the program code means for executing steps of the methodsdisclosed herein. The particular sequence of such executableinstructions or associated data structures represents examples ofcorresponding acts for implementing the functions described in suchsteps.

Those of skill in the art will appreciate that other embodiments of theinvention may be practiced in network computing environments with manytypes of computer system configurations, including personal computers,hand-held devices, multi-processor systems, microprocessor-based orprogrammable consumer electronics, network PCs, minicomputers, mainframecomputers, and the like. Embodiments may also be practiced indistributed computing environments where tasks are performed by localand remote processing devices that are linked (either by hardwiredlinks, wireless links, or by a combination thereof) through acommunications network. In a distributed computing environment, programmodules may be located in both local and remote memory storage devices.

The various embodiments described above are provided by way ofillustration only and should not be construed to limit the invention.For example, the principles herein may be applied to ordering almost anyproduct or service, such as pizza, groceries, dairy products, personalservices, hotel reservations, rental cars, and even vehicle purchases.Those skilled in the art will readily recognize various modificationsand changes that may be made to the present invention without followingthe example embodiments and applications illustrated and describedherein, and without departing from the true spirit and scope of thepresent invention.

We claim:
 1. A method comprising: receiving a search from a user;identifying, by a processor, a product category based on the search toyield an identified product category; presenting to the user a generalordering screen based on the product category; selecting, by theprocessor and from a plurality of speech recognition grammars, a speechrecognition grammar corresponding to the identified product category toyield a selected speech recognition grammar, the selected speechrecognition grammar recognizing commonly used words and phrasesdescribing a same item from different vendors in the identified productcategory; in response to an activation of the selected speechrecognition grammar by the processor, recognizing, by the processor, afirst user utterance to identify a vendor who offers items in theproduct category; in response to an activation of the selected speechrecognition grammar by the processor, recognizing, by the processor, asecond user utterance to identify a specific from the vendor; andplacing an order for the specific item with the vendor.
 2. The method ofclaim 1, further to sell the user additional items ancillary to thespecific item.
 3. The method of claim 1, further comprising: receivinguser billing information; and paying for the order using the userbilling information.
 4. The method of claim 1, further comprisingcommunicating to the user information regarding order pick up.
 5. Themethod of claim 1, further comprising: gathering delivery instructionsfrom the user; and communicating the delivery instructions to thevendor.
 6. The method of claim 1, further comprising retrieving a userprofile associated with the user.
 7. The method of claim 6, wherein theuser profile comprises an order history, the method further comprising:asking if the user wants to order a commonly ordered item from acommonly used vendor in the order history; and placing another order forthe commonly ordered item from the commonly used vendor if the userresponds positively.
 8. The method of claim 1, further comprisingdisplaying a graphical representation of the specific item on a screenof a wireless device.
 9. The method of claim 1, wherein a wirelessdevice receives the search from the user.
 10. The method of claim 1,further comprising verifying an identity of the user with speakerrecognition.
 11. A system comprising: a processor; and acomputer-readable, medium having instructions stored which, whenexecuted by the processor, cause the processor to perform operationscomprising: receiving a search from a user; identifying a productcategory based on the search to yield an identified product category;presenting to the user a general ordering screen based on the productcategory; selecting, from a plurality of speech recognition grammars, aspeech recognition grammar corresponding to the identified productcategory to yield a selected speech recognition grammar, the selectedspeech recognition grammar recognizing commonly used words and phrasesdescribing a same item from different vendors in the identified productcategory; in response to an activation of the selected speechrecognition grammar by the processor, recognizing a first user utteranceto identify a vendor who offers items in the product category; inresponse to an activation of the selected speech recognition grammar bythe processor, recognizing a second user utterance to identify aspecific item from the vendor; and placing an order for the specificitem with the vendor.
 12. The system of claim 11, the computer-readablemedium having additional instructions stored which, when executed by theprocessor, cause the processor to perform operations comprising offeringto sell the user additional items ancillary to the specific item. 13.The system of claim 11, the computer-readable medium having additionalinstructions stored which, when executed by the processor, cause theprocessor to perform operations comprising: receiving user billinginformation; and paying for the order using the user billinginformation.
 14. The system of claim 11, the computer-readable mediumhaving additional instructions stored which, when executed by theprocessor, cause the processor to perform operations comprisingcommunicating to the user information regarding order pick up.
 15. Thesystem of claim 11, the computer-readable medium having additionalinstructions stored which, when executed by the processor, cause theprocessor to perform operations comprising: gathering deliveryinstructions from the user; and communicating the delivery instructionsto the vendor.
 16. A computer-readable storage device havinginstructions stored which, when executed by a computing device, causethe computing device to perform operations comprising: receiving asearch from a user; identifying a product category based on the searchto yield an identified product category; presenting to the user ageneral ordering screen based on the product category; selecting, from aplurality of speech recognition grammars, a speech recognition grammarcorresponding to the identified product category to yield a selectedspeech recognition grammar, the selected speech recognition grammarrecognizing commonly used words and phrases describing a same item fromdifferent vendors in the identified product category; in response to anactivation of the selected speech recognition grammar by the computingdevice, recognizing a first user utterance to identify a vendor whooffers items in the product category; in response to an activation ofthe selected speech recognition grammar by the computing device,recognizing a second user utterance to identify a specific item from thevendor; and placing an order for the specific item with the vendor. 17.The computer-readable storage device of claim 16 having additionalinstructions stored which, when executed by the computing device, causethe computing device to perform operations comprising retrieving a userprofile associated with the user.
 18. The computer-readable storagedevice of claim 17, wherein the user profile comprises an order history,and having additional instructions stored which, when executed by thecomputing device, cause the computing device to perform operationscomprising: asking if the user wants to order a commonly ordered itemfrom a commonly used vendor in the order history; and placing anotherorder for the commonly ordered item from the commonly used vendor if theuser responds positively.
 19. The computer-readable storage device ofclaim 16 having additional instructions stored which, when executed bythe computing device, cause the computing device to perform operationscomprising displaying a graphical representation of the specific item ona screen on a wireless device.
 20. The computer-readable storage deviceof claim 16 having additional instructions stored which, when executedby the computing device, cause the computing device to performoperations comprising verifying an identity of the user with speakerrecognition.